® 



Europttteches P utonlamt 
European Patent Office 
Office europeen dec brevets 




® Pubficafion number 0 466 339 A2 



EUROPEAN PATENT APPLICATION 



© Application number 91305336-3 
® Date of filing: 14j0&91 



© tnta*:G06F 9/46 



<§) Priortty: 13.07-90 US 553203 

<§) Dated pubfication of application: 
1641.92 Bulletin 82/03 

® Designated Contracting States: 
DEFRGB 

© Applicant International Business Machines 
Corporation 
Old Orchard Road 



Afmont,N.Y.1Q504<US) 

© Inventor Dlsbrow, John Randolph 
16500 Sa Kennedy Road 
Los 08108.049503201$) 



© RepfBserrtative: Moss, Robert Douglas 
IBM United Kingdom Limited Intellectual 
Property Department Hursley Park 
Winchester Hampshire $021 2JN(GB) 



© A method of passing task messages In a data processing system. 



© Processors 5, 7 communicatively attached to a 
storage subsystem 3 place task messages for the 
subsystem on a queue: The piucessors do not have 
to wait on a queue lock set by anolher processor or 
sub-system whilst dequeuing a message. This is 
achieved by use of a double ended linked fist or 



queue of messages having an isolation/reference 
point wherein en enqueuing end of the fist Is toc- 
Icabte and accessible independently from the de- 
queuing end of the Ost The locking primitive may be 
of the mutti-proosssor lock synchronizing atomic 
type such as TEST AND SET. 
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Rek) of tho Invention 

This Invention relates to a method and means 
for passing messages between processors having 
order of magnitude speed differences to avoid the 
rate of message exchange being dominated by the 
lower speed. Such message passing occurs be- 
tween concurrently executing CPU's and an exter- 
nal storage sub-system (e.g. synchronous direct 
access storage device (DASD) array). 

Background of the Invention 

Contemporary high speed processing or super- 
computing conjures up the prospect of 1000 miBton 
instructiofts per 6econd (MIPS) of coordinate com- 
puting across multiple processors apertodicany re- 
ferencing substantially slower specialized proces- 
sors such as direct access storage device (DASD) 
array controllers. Syrtchnmization among proces- 
sors still requires a combination of locks and mes- 
sages. Locks serve to bind resources to tasks while 
messages and their processing operate as syn- 
chronizing events. In contemporary systems, task 
oriented messages are enqueued against re- 
sources. Also, the queued access is governed by a 
global lock. Thus, operations are paced by the 
slowest processor obtaining locked access to the 
queue, 

A single central processing unit (CPU) or pro- 
cessor typically includes a local operating system 
(OS), RAM implemented internal memory, tea) 
instruction and *data caches operatively formed 
from the interna] memory, an external store, and 
lock cache, and storage resource managers. How- 
ever, high speed or supercomputing involves ap- 
pScatkms executing over several processors. The 
appfications initiate tasks in the form of OS instruc- 
tions (READ/WRITE). These tasks are queued 
against the resources which process them. In this 
case, these are the general and special purpose 
processors of the high speed system. The tasks 
are relatively synchronized (ordered) with respect 
to each other by their position placement as mes- 
sages in the queue in a commonly accessible 
portion of processor shared internal memory. 

The messages (tasks) are expressed as encap- 
sulated operations defined over a range of ad- 
dresses. Where the messages relate to accessing 
external storage, they are enqueued by processors 
In the shared memory and await dequeuing and 
execution by the storage sub-system. Concurrently, 
messages Indicative of altered or completed stor- 
age access tasks are also enqueued by the storage 
sub-system in the shared memory and await de- 
queuing and execution by the proces so rs. By lock- 
ing the queue, the slowest proce sso r such as the 
external store (array controller) can pace the entire 



operation. 

As mentioned above, synchronization Is 
achieved among processors and tasks using 
queued access messages usually controlled by 

5 some form of locking. However, where there is a 
raw disparity in capacities, then much may go to 
waste while the fester processor awaits access to a 
queue currently bound (locked) to a substantially 
slower processor. 

10 Even where the processor engages external 
DASD storage without delay, there may be a gross 
mismatch of data rates. tttustrafiveJy, concurrent 
processors executing 100 MIPS and a 100 
MByte/sec data transfer rate might have to oommu- 

76 nicate with a gigabyte DASD having a 1 to 3 MByte 
transfer rate and a 10 millisecond access time. 

Patterson et al, * A Case for Redundant Arrays 
of Inexpensive Disks (RAID)", ACM SIGMOD Con- 
ference, Chicago Illinois, June 1-3, 1988 cBscusses 

20 the general solution with respect to data rate 
matching in the form of accessing N synchronized 
DASDs in parallel. Synchronous behavior requires 
N DASDs to rotate at the same rpm. have the 
same angular offset and be accessed In an fcfen- 

2s tical manner at the same time. 

As an alternative to communicating data in 
parallel via Patterson's synchronized DASD's, data 
rate mismatch has been managed by interlocks or 
lockable buffers. Buffer size and cost have re- 

30 mained as obstacles.,. ~. 

Beausoletl et al, USP 3336.582, 'Interlocked 
Communication System", issued August 15. 1967. 
shows an interlock over which a low speed proces- 
sor paces the transfer from a high speed proces- 

as sor. That is. a low speed processor such as a 
storage control unit (IBM 3880) strobes a 
CPU/channel (S/370) over a demand response in- 
terface indicating that It is available to process the 
next information unit 

40 Cage. USP 4.454395. "Buffer For Use With A 
Rxed Disk Controller", issued 6/12/1884, discloses 
a multi-ported random access memory managed 
by address register manipulation as an asynchro- 
nous partitioned circular buffer. Data is read from 

45 or written into consecutive RAM addresses on a 
parthion-at-a-time basis in wrap-around (circular) 
order. A partition/block consists of a fixed number 
of consecutive RAM addresses sized to hold a 
track sector of data, the RAM buffer having a 

so capacity of at least two such partitions. 

In Cage's buffer, the speed of trie movement of 
fixed blocks of data Is matched between the main 
memory (DMA) of a word processor and an at- 
tached DASD. A DASD write or read command 

55 results in data being moved either from the DMA or 
a DASD track sector into a first RAM partition. 
Because RAM operations are asynchronous, a de- 
mand transfer from a second RAM partition can be 
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overlapped wtth the first movement as an atomic 
part of the command (DASO read or wrtte) being 
executed. 

Knuth. The Art of Computer Programming', 
Second Edition, copyright Addison-Westey Pub. 
Co. 1968, 1973, Vol.1 Fundamental Algorithms, 
pages 234-239, 531-534, describes a "deque" as a 
Bnear fist tn which afl Insertions and deletions are 
made at the ends of the list He further defines an 
"input restricted deque" as a linear Gst in which 
(terns may be inserted at one end and removed 
from the other end (Seo2£.1 Exercise 1). 

Disclosure of the Invention 



The Invention provides a method of passing 
task messages In a data processing system Includ- 
ing a plurality of high-speed processors, an exter- 
nal storage subsystem, and shared Internal mem- 
ory, said method comprising the steps of. defining 
a first Inked fist in the shared internal memory with 
separately lockable first and second ends; and 
operating said Gst as a queue of task messages 
passirg between the high-speed processors and 
the storage subsystem, lockably enqueuing task 
messages at the first end of the queue and In- 
dependently lockably dequeuing task messages 
from the second end of the queue. 

Preferably tfie method further comprises the 
steps of defining a second finfced fist m the shared 
Internal memory with separately lockable first and 
second ends; and operating said second Gst as a 
queue of task messages passing between the high- 
speed processor s and the storage subsystem, with 
task messages being lockably enqueued by the 
external storage subsystem at the first end of the 
queue, and independently task messages being 
lockably dequeued by the high speed processors 
from the second end of the queue. Correspond- 
ingly, on toe first Gst task messages are enqueued 
at said first end of the queue by the high-speed 
processors, and dequeued from the second end of 
the queue by the external storage subsystem. 

In a preferred embodiment reference points 
and operator sets are defined over said first and 
second fists, one r ef erence point being associat e d 
with the second end of each fist, and each operator 
set inciuding top of Gst (TOQ) and bottom of list 
(BOQ) pointers, and top of list (TOQL) and bottom 
of list (BOQL) lockwords. A first lock Is obtained by 
one of the processors on said second end of the 
first Bst upon said one p rocessor matching the 
BOQL, whereupon task messages are embedded 
between the reference point and the last message 
mthelstendsaidfirstlockis released. A second 
lock can be obtained by Ihe subsystem on said 
first end of the first Gst upon said subsystem 
matching the TOQU allowing one or more task 



messages to be removed from said first Gst before 
said second lock b released. Only the holder of 
said second lock (TOQL) may altar any pointer to 
the non-reference endoftheGstoranyofthe task 
s messages In the Bst, and only the holder of the first 
lock (BOQL) may alter any pointer designating the 
reference point end of the Gst 

It is also preferred that the processors originate 
task messages in the form of control blocks 
w (DCB's) for the subsystem, each OCB specifying 
one or more access operations to be executed by 
said subsystem, said DC8*8 being enqueued on 
the first Gst by the originating processors, and said 
subsystem dequeuing and processing each DCB 
is from said first list according to either an external 
discipline (UFO, HL0, FIFO) or according to a 
priority reordering of said first fist or portion there- 
of. Each DCB assumes either a wafting, active, or 
completed status, the waiting DCB's constituting 
20 the first Gst. active DCB's being currently pro- 
cessed, and completed DCB's constitut in g the sec- 
ond Dst The subsystem then processes task mes- 
sages dequeued from said first Gst, updates each 
task message, and enqueues the updated task 
26 messages onto the second Gst said originating 
processor s dequeuing and process in g each up- 
dated task message from said second Gst accord- 
ing to either an external discipline (UFO. FILO, 
FIFO) or to a priority reordering of said second Gst 
30 or portion thereof . - w * ••■■y« - 

The Invention also provides a data processing 
system inducing a plurality of Wgtwpeed proces- 
sors, an external storage subsystem, and shared 
internal memory, said system further inducing: 
ss means for defining first end second finked nsts In 
the shared memory each with separately tockabte 
first and second ends; means for operating said 
first and second fists as wait and completion 
queues of task messages respectively, lockably 
40 enqueuing task messages at the first end of a 
queue and independently lockabty dequeuing task 
messages from the second end of the queue; 
means responsive to task messages originating 
from processors for writing the task messages into 
46 the shared memory, finking said task messages 
into the wait queue, and signaling the storage 
subsystem accordingly; means at the storage sub- 
system apertodicaliy responsive to said signalling 
for dequeuing the task messages from the wart 
so queue, actively processing the dequeued task mes- 
sages, enqueuing the processed task messages 
onto the completion queue In said shared memory, 
and signalling the processors accordingly; and 
means apertodlcaily responsive to the subsystem 
66 signals for dequeuing the processe d task mes- 
sages from the completion queue* 

The above method and means permit proces- 
sor of disparate speed to have overlapped access 
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id either add or remove messages In queued ac- 
oess (position order processing) relation to each 
other. Thus processors such as a CPU end an 
external storage subsystem placing a message on 
a queue no longer have to wait on a queue lock set 
by another processor or s u bsystem dequeuing a 
message. A double ended finked fist or queue of 
messages is used, preferably with an 
eolaton/reference point (NULL/BLANK). An en- 
queuing end of the fist (BOQ/BOQL) b lockable 
and accessible independently from the dequeing 
end of the fist (TOQ/TOQL). Typically the method 
and means utilize simple meta-processor lock 
primitives (in this regard, ~meta-processor~ de- 
notes available to all processors). The locking 
primitive may be of the mufti-processor lock syn- 
chronizing atomic type such as TEST AND SET. 
The processor obtains a lock, when available, be- 
fore inserting messages between the first end and 
the last message on the first queue, and then 
releases the lock. Likewise the subsystem obtains 
a lock, when available, before inserting messages 
between fhe first end and the last message on the 
second queue. The availability of a lock can be 
tested by comparison matching BOQL with the 
result from executing the lock primitive. 

Thus, in other words, task oriented messages 
can be passed between a plurality of high speed 
processors end a relatively low-speed external stor- 
age subsystem conruniinicativer/ coupled over a 
shared memory, using oppositely poled queues. 
First and second dense finked inear lists are de- 
fined in the shared memory, each fist having in- 
dependentty lockable first and second ends. A first 
lock is obtained by a processor on the first end of 
the first list when available, the processor inserting 
messages between the first end and the last mes- 
sage so finked, and releasing said first lock. The 
subsystem obtains another lock on the second end 
of the first fist when available, removing messages 
anywhere on the fist and releasing said other lock. 
On the first and second ends of the second fist the 
roles of the storage sub-ystem and processors are 
reversed. 

Brief Description of the Drawings 

Rgure 1 shows a CPU/ DASD array data flow 
emphasizing shared internal memory, high perfor- 
mance channel interface (HPCt), array controller, 
and DASD's. 

figures 2A-C respectively depict a waft queue, 
an active list and a completion queue of DASD 
control blocks <DC8's) selectabfy lockable at either 
end thereof. 

Figure 3 illustrates the enqueue operation on a 
DCB wait and completion queue. 

Rgure 4A-B sets forth the dequeue operations 



on a respective first and second example of a DC8 
wait and completion queue. 

Detailed Description 

6 

The preferred embodiment of this invention 
uses a high speed murfrprocessor host interacting 
with a slower external store. The external store is 
illustratively expressed as a synchronous array of 
to N DASD's and an array controller. To enhance 
appreciation of this form of external storage, a brief 
description of data organization on the array 
(striping) and the use of information redundancy 
(parity blocks, ECC) is provided. 
16 Patterson's type 3 DASD array synchronously 
reads and writes to N OASOs in column major 
order. However. N-1 of the DASD's contain data 
and one DASD contains a parity ranging over the 
other data OASOs. That is, one check DASD is 
20 provided for the group. The contents of the failed 
DASD can be reconstructed in the manner of 
Ouchi. US Pat 4,092.732. -System for Recovering 
Data Stored in a Failed Memory Unit", issued May 
30, 1978, which dsdoses the spreading of data 
blocks from the same logical file across a string of 
N- 1 DASD's and recording a parity block on the 
Nth DASD. The parity block is an XORing of the 
parity contents of the N-1 other blocks. Contents 
from any single inaccessible DASD can be recov- 
ered by XOfling the parity blocks with the blocks 
stored on the N-2 remaining accessible DASD's. A 
similar result can be achieved if the parity blocks 
are not available. . 

Typically in block oriented data, a parity suffix 
or equivalent (Hamming, CRG) is appended to 
each data block. Thus, each parity suffix can be 
invoked to detect/correct intra-block error. As de- 
scribed in Ouchi, when one or more of the blocks 
of an N-1 sequence are unavailable, a parity block, 
which a priori spans the N-1 block sequence, is 
used in conjunction with the remaining blocks to 
rebuild the unavailable data block. Efficient codes 
per se (Hamming, Cyclic Redundancy Check, 
Reed-Solomon) are elsewhere treated in the litera- 
ture. 

Referring now to Rgure 1. there is shown a 
system including a an array of processors 1 cou- 
pling as externa) storage a synchronous DASD 
array 3. The processors 5, 7, 9 are of the high 
performance variety such that when operated con- 
currently they have processing speeds in the order 
of 1004- MIPS. A slower special purpose processor 
in the form of DASD array 3 is coupled to array 1 
by way of adapter 15. Other Irformafion processing 
sources or sinks such as local area networks, print- 
ers, or displays would likewise be coupled over 
counterpart adapters 17. 

The fast and stow processors communicate 
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within array 1 over every highspeed bus 11 udng 
a portion of shared Internal RAM 13 as a special- 
bed message repository. The processors constitute 
a distributed peer coupled system with no central- 
ized operating system or process control. AO re- 
sources such as interna) memory 13, interrupt fa- 
cifity. and global registers (not shown) are available 
to any proces s or including the externa) storage or 
DASD array subsystem (DAS) 3. 

DAS 3 preferably comprises a RAIDS type 
DASD array and an associated array controller as 
described for example In the Patterson reference 
and tn the co-pencTng Brady et el. application, EP 
91 304503-5, -METHOD AND MEANS FOR AC- 
CESSING OASD ARRAYS WITH TUNED DATA 
TRANSFBt RATE AND CONCURRENCY", 
(App&canfs ref. no. SAM0028). Even though DAS 
operates in a peer-coupled relationship with the 
other p rocessors, it nevertheless queues task ori- 
ented messages against other processors or re- 
sources. Any change In the queue or other action 
requests such as halt current operations are in- 
cficated fay way of special purpose signals (lap 
signals). 

OAS receives tasks as messages and commu- 
nicates the results of task processing via control 
blocks arranged in the form of a dedicated queue. 
This queue, as exBmpBfied tn figure 2, remains in 
host 1 1nterna! memory 13. The system includes a 
facility to reorder the queue according to changing 
systBm priorities. 

Referring 8gain to Rgure 1, there Is shown an - 
array controller (elements 1*31) coupling the host 
bus 11 by way of channel adapter 15. This path 
provides access to aD systBm resources Including 
memory 13. global registers, end tap signals on 
behalf of OAS 3. Adapter 15 preferably attaches 
DAS over a pair of simplex megabyte rate receive 
and transmit interfaces 19, 21. The interfaces, 
known also as High Performance Parallel Interfaces 
(HIPPO, are deserted tn the ANSI Draft Standard 
of 8/29/1889. X3TB/88-127, Revision 64. This facili- 
tates receipt of the so-caned tap signals from the 
host and access to the host as initiated by DAS. 

Host or system functions available for DAS use 
include READ/WRITE internal memory 13, atomic 
TEST AND SET for lock operations in internal 
memory 13, atomic operations on global registers, 
receipt and Interpretation of tap signals from other 
processors or system elements, and the generation 
of tap signals to the system or host 

OAS 3 operates one addressable array of 
DASD's 33, 35, 37 synchronized via sync circuit 39 
to each rotate at the same rpm, have the same 
angular offset, and be accessed In 8n identical 
manner at the same time. This solution maximizes 
data transfer rate. This permits high speed sequen- 
tial or skip sequential DASD data transfer. 



Data movement In the Host/DAS direction 
starts from Internal memory 13 over bus 11 
through adapter 15, receiver 19, encfing tn striping 
and parity logic 23. Logic 23 includes the neces- 
s sary cfigHal and timing circuits to calculate a parity 
block by XORing the N-1 data blocks and transfer- 
ring counterpart blocks to ones of the buffet and 
ECC circuits 27, 29, and 31. Each block is also 
protected by appending an ECC byte thereto for 
to error detection end correction on an intrerbkxk 
basis. Access to the N OASO^e Is made synchro- 
nously in the conventional manner. The obverse 
obtains when data movement proceeds In the 
DAS/Host direction. 
f6 The task or request Is defined in a control 
block termed a OASD Control Block or DCS. A 
processor at the host builds a DAS DCS in internal 
memory 13. It then Enks the DCS to a WATT queue 
for DAS and signals the enqueuing operation. Sub- 
20 sequentfy, DAS moves the DCB onto the ACTIVE 
UST, performs OCB requested functions, updates 
the DCS and places it on the COMPLETION 
queue. After detecting the I/O completion, the host 
dequeues the updated DCS from the COMPIE- 
25 TTON queue and ascertains the outcome of the 
DC8 specified operation. 

Restated, a DAS I/O request Is responsive to 
any processor 54 invoking a READ/WRITE from 
Its OS. A DCB is buift and placed on a WAIT 
so queue located In internal memory 13. A tap signal 
Is then sent to OAS. In turn, the DAS inspects the 
queue for the next queued DC8 using any one of a 
number of work management algorithms (RFO. 
> RLO, UFO etc). OAS does not need to respond to 
36 the tap signal as a priority event - 

Referring now to Figures 2ArC, the system Is 
arranged such that the DAS "task WATT queues* 
are priority ordered A request may be relocated in 
the queue relative to other tasks or DC8*s at any 
40 time prior to its being made active by the DAS. In 
this invention, such reordering Is a spedal fundfon 
of a DEQUE operation to be explained subse- 
quently. Note, that the DCB's remain at the same 
internal memory 13 real address irrespective of 
48 changes in queue finking priority or active/waiting 
status. 

One of the attributes of peer-coupled proces- 
sors is that the DAS is responsive to receipt either 
of a tap signal or its work management algorithm 

go by finding the next queued DCB via reading an 
anchor pointer tn internal memory 13 established 
during Wtlattffltton and changing the first waiting 
DCB status from -WAITlng" to -active. This Is 
accomplished by moving the DCB show in Figure 

6s 2A to the ACTIVE UST per Rgure 2B. This is 
brought about by the OAS executing a sequence of 
internal memory access operations. Once the DCB 
is "active", the DAS processes the DCB according 
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a function code cont ai ned in said DCB. Retatedly. 
address Information in the DCB define tha extant of 
data to be transferred. In this regard, the data 
transfer is effectuated by DAS initiated operations 
through the channel adapter 15. 

It should be appreciated that each OC8 can 
assume one of three states, namely, "active", 
drafting", or "completed** 

When a data transfer operation ts completed, 
the OAS writes completion status in the OC8 t 
changes the DCB state as per Rgure 26 from 
"active" to "completed", and enqueues the DCB 
on the assoc iat ed completion queue per Figure 2C. 
Also, the DAS may execute a number of OCB 
specified completion notification primitives. Signifi- 
cantly, tha transfer of data to and from internal 
memory 13 and the DAS is under DAS control 

Referring still to Figures 2A-C, there are shown 
several queues of DCB's. Each queue element 
genericafly called QEL. Is a contiguous set of 
words in a memory which both processor classes 
can fetch from and store to in shared memory 1& 
A QEL includes message words as weO as a ink 
pointer word required for queuefog. Although each 
QEL could be a different length, advantageously a 
simple system might use uniform steed QELs of. 
perhaps. 32 words each. Typically, each shared 
memory word itself consists of at least enough bit 
positions to contain the address of any other word. 
For example each word of shared memory could 
be 64 Ms long. J - 

A pointer to a QEL is the address of one of the 
words In the QEL (as used in this specification and 
figures 2A-C, the term "pointer* refers either to an 
address, or to a place where such an address is 
found). Context determines which is meant The 
word pointed to, the Gnk pointer word, normally 
contains the address of another QEL but can con- 
tain a predetermined null value such as aD bits off, 
or an on* 

As illustrated Figures 2A-C, a Blank-QEL is a 
QEL whose Gnk pointer word is null Also, a queue 
may be formed from either a single Blank-QEL or 
a set of QELs finked one to another by pointers 
leading from a top QEL to a bottom QEL The latter 
is always a Blank-QEL when the bottom pointer is 
unlocked. 

Queues are locked by using a multiprocessor 
synchronizing lock operation available to all partici- 
pating multiprocessor classes. An example is the 
classic Test and Set operation which stores a lock 
constant In a word of shared memory. H the pre- 
ceding fetch, wttich is part of the same atomic 
operation, returns a value other than this lock con- 
stant then the Test and Set opera ti on has success- 
fully locked me lock word. 

A queue header consists of four words in the 
shared memory as shown following: 



TOO Top of Queue Pointer Shared mem- 
ory address of Ink pointer word of the 
top QEL in the queue. 

BOG Bottom of Queue Pointer Shared 
& memory address of Gnk pointer word 

of the bottom QEL in the queue. 

TOOL TOQ Lock Word. For example a Test 
and Set lock word. Only the holder of 
the TOQ lock may alter the TOQ 
10 pointer or any of the contents of 

QELs linked above the QEL pointed 
to by BOO. (Note that the BOQ point- 
er may change while TOQ is locked; 
see BOOL) 

is BOOL BOQ Lock Word. Only the holder of 
the BOQ lock may alter the contents 
of the QEL at the bottom of the 
queue, possibly making it non-blank. 
Only the BOQL processor/holder may 
20 change the BOQ pointer value, and 

then only, if the new value is the ad- 
dress of a valid Blank-QEL 
Changing the BOQ pointer moves tha isolation 
point between the enqueuing and dequeuing pre- 
ss cesses. Once the BOQ is changed, the BOQ lock 
holder's authority to update is reduced to the New- 
Blank-QEL and to the BOQ pointer. A TOQ lock 
holder is always free to update any QELs down to, 
but not Including, the QEL pointed to by the BOQ, 
so whether the BOQ is locked or not ,. 

Applying the above definitions to figures 2A-C, 
there are shown, constructs indicative of tiie three, 
states into, which each DCB must be resolved (wart, 
active, completed). 'DOB'S classified as waiting or 
ss completed are enqueued while an active DCB is 
being processed. In each queue, each DCB has a 
pointer to the neod 0C8 in the chain. The BOQ 
points to a nuQ or blank DCB operative as the 
queue bottom while TOQ points to the most recent 
40 queue addition. 

Referring now to figure 3, there is depicted an 
enqueue operation as the loading of a message 
into the Blank-QEL at the bottom of a queue, and 
the appending to the bottom of the queue of an 
45 additional QEL as the New-Blank-QEL The en- 
queue operation includes the steps of: 

1. locking BOQ, conditionally waiting for the 
queue to become available; 2. writing a null into 
Dnk pointer of the additional QEL *New-8lank"; 3. 
so writing the memory address of New-Blank into 
linkage Pointer of Old-Blank: 4. loading a message 
into the former Blank-QBL, *OkS- Blank"; 5. putting 
memory address of New-Blank into the BOQ point- 
er word; and 6. unlocking BOQ. 
« The original queue has not been altered except 
for the contents of the original Blank-QEL The 
original Blank-QEL was initially at the bottom of the 
queue. Only the processor/holder of the TOQ lock 
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may attar the queue's non-blank QELs. 

The enqueuing processor can add more than 
one QEL tn preparation for enqueuing, the addi- 
tional QELs are to be (inked one to another from 
Top-AddnKlEL to Bot-Addnl-QB- The steps are o 
listed below tn an order which continuously main- 
tains a weO formed queue. Note, a "well formed 
queue" refers to a TOQ-BOQ pair and its asso- 
ciated queue conforming to the above definitions. 

1. locking BOQ, perhaps waiting for ft to be- io 
come available; 

2. putting nuO Into Bnk pointer of Bot-AddnttEL, 
making ft New-Blank; 

3. loading messages into Old-Blank, and all 
Addnfr-QELs except New-Blank; f 6 
4w putting memory address of Top-Addnl into 

fink pointer of Old Blank: 

6. putting address of New-Blank into BOO; and 

a unlocking BOQ. 

Referring now to Figures 4A-B, there are de- 20 
pfcted two examples using a DEQUE operation in 
which a single QB. is removed from fl*e top of a 
queue. A processor copies into the TOQ the con- 
tents of the link pointer from the first QEL in the 
queue. To remove a continuously linked sequence 2s 
of QELs, a processor changes tfte TOQ pointer, or 
a single QEL Gnk pointer, so as to point around the 
QELs being removed. The dequeue steps ere Ist- 
ed below In an order whfch continuously maintains 
a well formed queue/' l *o 

1. if TOQ °' BOO, exiting without locking as 

queue Is empty of non-blanks; 

Z locking TOO. corvfifionaDy after waiting for ft 

to become avsflabte: " : < 

a reading BOQ. any QEL at that address or ss 

linked beyond it Is ineligible; 

4. removing one or more eCgiMe QELs from 
anywhere on the queue; and 

6. unlocking TOO. 

The QEL pointed to by BOQ. when BOQ is 40 
read, is treated as ineligible. While a BOQ lock 
holder may make this a valid QEL and may change 
BOQ, the QEL will remain irtefigible for this execu- 
tion of enqueue or reorder. 

Reordering is a two step process In which 46 
QEL(s) are first dequeued from, and then reinser- 
ted into, the eligible QEL chain. The processor 
doing the reordering holds the TOQ lock for the 
duration of both steps. 80 to move a single QEL, 
called "MovingOEL*, to the top of the queue, a so 
proce sso r s dequeues the QEU as defined pre* 
vlousty, and then reinserts ft by changing Movtng- 
QEL's Bnk pointer to the value tn TOQ, and putting 
the address of Movlng-QEL Into TOO. 

tn a stmflar manner e proce s sor can dequeue 65 
any number of eligible QELs, Gnk them together in 
one or more fragmentary chains, and then reinsert 
the fragments back into the queue. Reinsertion can 



be done maintaining a weO formed queue by: 

1. setting Ink pointer of tragmenfa final QEL to 
address of QEL that wtn foOow fragment in 
queue; and 

2. setting TOQ pointer, or Gnk pointer of QEL 
that is to precede fragment to the address of 
first QEL in fragment 

For any operation to continuously maintain a 
well formed queue, each single word storage up- 
date of a pointer in shared memory must complete 
fully. H a process then follows the steps of an 
operation in the order given, the queue wiO remain 
wen formed, In the event of most hardware failures, 
or of a premature process termination, some in- 
transit QELs may not be on the queue, but the 
queue itself will remain wefl formed Other pro- 
cesses wiO be able to continue working with such a 
queue. 

In the initial example the allocation of shared 
memory can be a processor only function. This can 
be achieved by having the controllers reuse each 
work queue element as the QEL in which to report 
completion status. 

Claims 

1. A method of passing task messages in a data 
processi/xj system inducting a plurality of high- 
speed processors (5, 7), an external storage 
subsystem (3), and shared internal memory 
(13), said method comprising the steps of. 

defining a first linked Ost in the shared 
internal memory with separately lockable first 
and second ends; and 

op e r ating said Ost as a queue of task mes- 
sages passing between the highspeed proces- 
sors and the storage subsytem, lockaWy en- 
queuing task messages at the first end of the 
queue and independently lockabty dequeuing 
task messages from the second end of the 
queue. 

2. A method as claimed in claim 1. in which the 
task messages are enqueued at said first end 
of the queue by the high-speed processors, 
and dequeued from the second end of the 
queue by the external storage subsystem. 

3. A method as claimed in claim 2, further com- 
prising the steps oh 

defining a second linked list in the shared 
Interna) memory with separately lockable first 
and second ends; and 

operating said second list as a queue of 
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task messages passing between the high- 
speed processors and the storage subsystem, 
task messages being tockabty enqueued by 
the external storage subsystem at Che first end 
of the queue, and independently task mes- 
sages being tocfcabty dequeued by the high 
speed processors from the second end of the 
queue. 

4. A method as claimed in claim 3, further com- 
prising the steps of: 

defining reference points and operator sets 
over said first and second fists, one reference 
point being associated with the second end of 
each fist, and each operator set including top 
of 6st (TOQ) and bottom of fist (BOG) pointers, 
and top of 1st (TOOL) and bottom of Gst 
(BOOL) todcwords; 

obtaining a first lock by one of the proces- 
sors on said second end of the first list upon 
said one processor matching the BOOL em* 
bedding task messages between the reference 
point and the last message in the Gst and 
releasing said first lock: 

obtaining a second lock by the subsystem 
on said first end of the first Gst upon said 
subsystem matching the TOQU removing one 
or more task messages from said first Gst and 
releasing said second lock. 

& A method as claimed in claim 4, wherein only 
the holder of said second lock (TOOL) may 
alter any pointer to foe non-reference point end 
of the Gst or any of the task messages in the 
fist and further wherein only the holder of the 
first lock (BOQL) may alter any pointer des- 
ignating the reference point end of the fist 

& A method as claimed in any preceding claim, 
wherein said method further comprises the 
step of: 

reordering the Gst of task messages by 
obtaining a lock on the second end of the first 
list when available, removing task messages 
from anywhere on the fist finking at least some 
of the dequeued task messages together In 
one or more fragmentary chains, and then rein- 
serting the fragmentary chains back Into the 
queue, and releasing the lode 

7. A method as claimed in any of claims 2 to 6. 
wherein the processors originate task mes- 
sages in the form of control blocks (DCB's) for 
the subsystem, each DC8 specifying one or 



more access operations to be executed by 
said subsystem, said DC8*s being enqueued 
on the first fist by the originating processors, 
and said subsystem dequeuing and process in g 
s each DCS from said first fist according to 
either an external discipline (UFO, FllO. FIFO) 
or accoftfing to a priority reordering of said 
first Gst or portion thereof. 

io & A method as claimed in claim 7, wherein each 
DCS assumes either a waiting, active, or com- 
pleted status, the waiting 0C8*s constituting 
the first fist active DCB's being currently pro- 
cessed, and completed DCB's constituting the 

is second Gst 

9l A method as claimed in claim 7 or 8, wherein 
the subsystem processes task messages de- 
queued from said first Gst updates each task 

20 message, and enqueues the updated task 
messages onto the second Gst said originating 
processors dequeuing and processing each 
updated task message from said second fist 
according to either an external discipline (UFO. 

25 F1LO, FIFO) or to a priority reordering of said 
second fist or portion thereof. 

10. A data processing system including a plurality 
of high-speed processors (5, 7), an external 
30 storage subsystem (3), and shared internal 
memory (13). said system further including: 

means for defining first and second (inked 
fists in the shared memory each with sepa- , 
35 rately locfcabJe first and second ends; 

means tor operating said first and second 
lists as wait and completion queues of task 
messages respectively, tockabty enqueuing 
40 task messages at the first end of a queue and 
independently tockabfy dequeuing task mes- 
sages from the second end of the queue; 

means responsive to task messages origi- 
ns rating from processors for writing the task 
messages into the shared memory, finking said 
task messages into the wait queue, and signal- 
ting the storage subsystem accordingly; 

so means at the storage subsystem 

apertodlcaiiy responsive to said signalling for 
dequeuing the task messages from the wait 
queue, actively processing the dequeued task 
messages, enqueuing the processed task mes- 

65 sages onto the completion queue in said 
shared memory, and signalling the processors 
accordingly; and 
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means 8p6fto(ficany responsive to the sub- 
system signals lor dequeuing the processed 
task messages from the completion queue. 
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